Journal of Education and Practice 

ISSN 2222-1735 (Paper) ISSN 2222-288X (Online) 

Vol.7, No. 10, 2016 


www.iiste.org 


JUil 

i$te 


A Comparison Between Students’ Performance In Multiple 
Choice and Modified Essay Questions in the MBBS Pediatrics 
Examination at the College of Medicine, King Khalid University, 

KSA 


Karimeldin M A Salih 1 , Mohamed Abdullah Al-Gosadi Alshehri 2 , Omer Abdelgadir Elfaki 3 

1. King Khalid University, College of Medicine, Department of Pediatrics, Medical Education 

2. King Khalid University, College of Medicine, Head department of Pediatrics 

3. King Khalid University, College of Medicine, Department of Medicine, Medical Education 

Abstract 

Objectives : To investigate the relation between the students' scores in MCQs and MEQs of the summative 
assessment in pediatrics at the College of medicine KKU. 

Introduction : Student assessment is the most difficult task in medicine since it is ultimately related to human life 
and safety. Assessment can take different types of formats with advantages and disadvantages and there is no 
single prescribed method of choice. 

Research Methods : This is a quantitative, observational retrospective study enrolling 50 students who completed 
the final MBBS examination in pediatrics in 2015.The medical students’ scores in MCQs and MEQs in the final 
MBBS pediatrics examination were collected and SPSS used for analysis. 

Results : 52% of the students scored A and B, 46% scored C and 2% scored D and nobody scored F. The mean 
percentage scores in MCQs, MEQs and the combined are75.03%, 79.99% and 76.68% respectively. The 
standard deviation (SD) for MCQs, MEQs and combined are 0.0753, 0.0763 and 0.0559 respectively.There are 
no significant differences in means and SD. The correlation (CR) between MCQs and MEQs is -1.73%. The CR 
between MCQs and MEQs for those who scored A and B is 7.38% which could indicate a weak but significant 
+ve CR (p value 0.017), (table3). For those who scored C inverse CR (-74.38%) was observed between MCQs 
and MEQs. 

Discussion : The main finding of this study was the weakly negative CR between students' performance on 
MCQs and MEQs. Good performance in MCQs is not necessary correlated with good performance in LEQs. 

This is not in agreement with Oyebola et al. and Moqattash et al findings, who reported positive CR in the 
performance of their students. The discrepancy observed in performance between MCQs and MEQs could be 
explained by the subjectivity and the difficulty in avoiding bias in setting and marking essay questions. 
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1. Literature review 

Student assessment is the most difficult task in medicine since it is related to human life and safety m . 
Assessment is an active, continuous process and is the corner stone in medical education. It could determine 
students’ approaches and perception towards their learning and study 12 ’ 71 . The significance of MCQs as a tool for 
assessment is well recognized due to its validity, reliability, ability to be used for large numbers of students and 
easy marking using a scanner or a computer. Moreover, MCQs can assess knowledge, cognition and can cover 
large topics in the curriculum. However, MCQs also have some disadvantages which include, need of experts to 
construct them, encouragement of superficial reading by students and the misleading of some clever students 
who might go in more than the needed depth for answering 18 9) . In contrast to MCQs, essay questions allow 
students to go in depth and reflect themselves well l101 . Moreover, short essay questions are sensitive and can 
allow a student to prove his skill in high cognitive level but consume more time and might open doors for bias 
should it not be well constructed 111 '. MCQs ability to anticipate the aggregate performance is considered low 
compared to essay questions tl2> . In contradistinction, Day et al considered the essay questions to be of low 
sensitivity in measuring performance compared to MCQs (I3) . Despite this debate the ideal way is to use multiple 
methods when conducting assessment (1) . This is essential for medical schools to be accountable to their 
stakeholders. 
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The department of pediatrics in the College of Medicine KKU uses MCQs and MEQs for long time. The 
department is interested in comparing the performance in these two methods especially in the absence of such 
study in a similar context. Thus, the objective of the current study was to investigate the relation between the 
students' scores in MCQs and MEQs of the summative assessment in pediatrics. 

Theory and hypotheses 

Null hypothesis: there is no relation between student performance (scoring) in MCQs and MEQs in 
pediatrics, college of medicine, KKU. 

Alternative hypothesis: there is a relation between student performance (scoring) in MCQs and LEQs in 
pediatrics, college of medicine, KKU. 


2. Research methods 

This is a quantitative, observational retrospective study enrolling 50 students who completed the final MBBS 
examination in pediatrics in June 2015.The medical students’ scores in MCQs and MEQs in the final MBBS 
pediatrics examination were collected. These scores are kept in the department coordinator computer which is 
secured and protected by a pass word only known to the coordinator. The data used includes only the scores 
without any personal identification information of any student. 

KKU College of medicine was established in 1980. The MBBS program of the college can still be described as 
discipline-based. The pediatrics course is taught in the 5th and 6th years with final examination in pediatrics as 
pre-requisite for graduation. The total mark is 45 (For this component MCQs carrying 30 marks, MEQs carrying 
15 marks) and the grading system adopts letter grading as follows: A for score above 40 marks, B from 35-39,C 
30-34, D 25-29 and F which is failure below 25. 

Both MCQs and MEQs were designed to cover the pediatrics syllabus in the curriculum with well-planned 
blueprint to ensure validity and well distribution of the questions according to the weight and importance of the 
topics in the curriculum. The MCQs component of the final examination consisted of 60 questions in form of the 
best of four options. The time allowed for the examination was 1.5 hours for the 60 MCQs (1.5 minutes for each 
MCQ).The MEQs contained three open-ended questions including two clinical cases (easel and case 2) and one 
topic from the common pediatrics problems. Especial emphasis was made on interpretation; layout of differential 
diagnoses, case summary and organization of the information. 

SPSS was used for data analysis. Pearson's correlation coefficient (r) was used to compare students' performance 
(scoring) in MCQs and MEQs. T-test was used to test the significance of differences in performance in MCQs 
and MEQs. The level of significance was set at P < 0.05. The analysis was done with the help of the statistician 
of the department of medical education in the college. 


3. Results 

52% of the student scored A and B, 46% scored C and 2% scored D and nobody scored F (Table 1, Fig. 1). The 
mean of the percentage score in MCQs, MEQs and the combined score are75.03%, 79.99% and 76.68% 
respectively with no significant difference. The standard deviation (SD) for MCQs, MEQs and the combined 
scores are 0.0753, 0.0763 and 0.0559 respectively (Table 2). The CR between MCQs and MEQs is -1.73 where 
there is weak negative correlation. This indicated that good achievement in MCQs would not necessarily be 
associated with good achievement in MEQs and vice versa. Not only that, but it further indicated that high 
performance in one is associated with low performance in the other. This finding made it necessary to investigate 
the relationship between both MCQs and MEQs across the different grades. This showed that the CR between 
MCQs and MEQs for those who scored A and B is 7.38% indicating weak +ve but significant CR (p value 
0.017), (Table3). For those who scored C inverse correlation was observed between MCQs and MEQs (CR of - 
74.38). For F, and D grades there were no enough scores. 
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Table 1: Grade distribution 


Grades 

freq. 

% 

A&B 

26 

52 

C 

23 

46 

D 

1 

2 

F 

0 

0 

Total 

50 

100 


'I 

■$T| 


Table 2: Descriptive statistics 


Exam component 

Mean (in %) 

S.D 

MCQs (out of 30) 

75.03% 

0.0753 

LEQs (out of 15) 

79.99% 

0.0763 

Combined (out of 
45) 

76.68% 

0.0559 
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Figure 1: Grades distribution 


Table 3: Comparison between MCQs & MEQs 


Statistics 

A&B 

grades 

C grade 

Combined 

Correlation 

7.25 

-74.37 

-1.73 

p-values 
(by t test) 

0.017 

0.0144 

0.001 
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4. Discussion 

To the best of our knowledge, this study is the first one to be performed in a Saudi medical school context. The 
study aim was to investigate the relation between the students' scores in MCQs and MEQs of the summative 
assessment in pediatrics at the College of Medicine KKU.The main finding of this study was the weakly 
negative correlation between students' performance in MCQs and MEQs. The good performance in MCQs was 
not necessarily correlated with good performance in MEQs. This negative correlation between the performance 
in MCQs and MEQs does not agree with the findings of Oyebola et al. <14> and Moqattash et al. (15) who reported 
positive correlation in the performance of their students. This difference and disagreement could be explained by 
the difficulties in avoiding bias and subjectivity in setting and marking essay questions. The positive correlation 
in performance between MEQs and MCQs for those who achieved A and B could be easily explained since the 
good performers logically perform well in any examination format as suggested by Anbar (9) . The higher scoring 
in MCQs could be explained by the possibility of guessing and the presence of recall questions, while bias and 
subjectivity in marking of MEQs could lead to low scoring in MEQs. This might explain the negative 
correlation observed in C and D grades. But, the observation itself is not constant in all previous similar 
studies 16 ’. One of the factors that might affect genaralizability of these results might be the low number of the 
enrolled students and the unequal distribution of marks between MCQs and LEQs. 


5. Conclusion 

There was weak negative correlation between the students’ performance in MCQs and MEQs. This correlation 
was not the same among the different grades. 


6. Recommendation 

Review of the MCQs and MEQs items used in this summative assessment is essential for improvement. Use of 
other methods of assessment should be considered. 
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